Computer-Assisted Language Comparison: State of the ArtW

Historical language comparison opens windows onto a human past, long before the availability of written records. Since traditional language comparison within the framework of the comparative method is largely based on manual data comparison, requiring the meticulous sifting through dictionaries, word lists, and grammars, the framework is difficult to apply, especially in times where more and more data have become available in digital form. Unfortunately, it is not possible to simply automate the process of historical language comparison, not only because computational solutions lag behind human judgments in historical linguistics, but also because they lack the flexibility that would allow them to integrate various types of information from various kinds of sources. A more promising approach is to integrate computational and classical approaches within a computer-assisted framework, “neither completely computer-driven nor ignorant of the assistance computers afford” [1, p. 4]. In this paper, we will illustrate what we consider the current state of the art of computer-assisted language comparison by presenting a workflow that starts with raw data and leads up to a stage where sound correspondence patterns across multiple languages have been identified and can be readily presented, inspected, and discussed. We illustrate this workflow with the help of a newly prepared dataset on Hmong-Mien languages. Our illustration is accompanied by Python code and instructions on how to use additional web-based tools we developed so that users can apply our workflow for their own purposes

Computer-Assisted Language Comparison: State of the Art

Historical language comparison opens windows onto a human past, long before the availability of written records. Since traditional language comparison within the framework of the comparative method is largely based on manual data comparison, requiring the meticulous sifting through dictionaries, word lists, and grammars, the framework is difficult to apply, especially in times where more and more data have become available in digital form. Unfortunately, it is not possible to simply automate the process of historical language comparison, not only because computational solutions lag behind human judgments in historical linguistics, but also because they lack the flexibility that would allow them to integrate various types of information from various kinds of sources. A more promising approach is to integrate computational and classical approaches within a computer-assisted framework, “neither completely computer-driven nor ignorant of the assistance computers afford” [1, p. 4]. In this paper, we will illustrate what we consider the current state of the art of computer-assisted language comparison by presenting a workflow that starts with raw data and leads up to a stage where sound correspondence patterns across multiple languages have been identified and can be readily presented, inspected, and discussed. We illustrate this workflow with the help of a newly prepared dataset on Hmong-Mien languages. Our illustration is accompanied by Python code and instructions on how to use additional web-based tools we developed so that users can apply our workflow for their own purposes

The Database of Cross-Linguistic Colexifications, reproducible analysis of cross-linguistic polysemies

Advances in computer-assisted linguistic research have been greatly infuential in reshaping linguistic research. With the increasing availability of interconnected datasets created and curated by researchers, more and more interwoven questions can now be investigated. Such advances, however, are bringing high requirements in terms of rigorousness for preparing and curating datasets. Here we present CLICS, a Database of Cross-Linguistic Colexifcations (CLICS). CLICS tackles interconnected interdisciplinary research questions about the colexifcation of words across semantic categories in the world's languages, and show-cases best practices for preparing data for cross-linguistic research. This is done by addressing shortcomings of an earlier version of the database, CLICS2, and by supplying an updated version with CLICS3, which massively increases the size and scope of the project. We provide tools and guidelines for this purpose and discuss insights resulting from organizing student tasks for database updatesTT, MSW, NES, YL, and JML were funded by the the ERC Starting Grant 715618 Computer-Assisted Language Comparison (http://calc.digling.org). SJG was supported by the Australian Research Council’s Discovery Projects funding scheme (project number DE 120101954) and the ARC Center of Excellence for the Dynamics of Language grant (CE140100041). MKT was supported by the Riksbankens Jubileums Fond (Grant SAB17-0588:1). TB was funded by the Swiss National Science Foundation, P2BEP1_181779, “Reconstruction of Proto-Western Kho-Bwa”

Enhancing morphological annotation for internal language comparison

In language comparison, there is a long history of using concept-based wordlists to get insights into the degree of similarity between languages, going back at least to Morris Swadesh (Swadesh 1950). For these purposes, words from different languages that share the same meaning are compared, either manually or with computational methods. The latter have the advantages of being both faster and more consistent. However, there are also limits to what computer-based methods can detect for the tim..

Biological metaphors and methods in historical linguistics (2): Words and genes

As was mentioned in the introduction to this series of blogposts, both species and languages are often presented in a tree model. In biology, trees of each individual gene are created in order to account for horizontal transmission and other processes in which the history of a gene differs from the general history of its genome. From the sum of these trees, the species trees are then derived, a method called gene tree reconciliation (Nakhleh 2013). In linguistics on the other hand, phylogenet..

Biological metaphors and methods in historical linguistics (3): Homology and homoplasy

As we have seen in previous instances of this blog post series, there are many parallels but also many differences between the evolutionary branches of biology and of linguistics. In the following, I will present a comparison of the causes due to which two related inheritable entities (e.g. two words or two genes of different languages or species) may differ from each other, or two unrelated ones resemble each other. The linguistic categories presented here can also be found in List (2016) wh..